Taking the Measure of Ideology: Empirically Measuring Supreme Court Cases

INTRODUCTION Although the tools of empirical legal analysis have become increasingly sophisticated, much of this development has been asymmetrical. Empiricists have developed numerous ways of assessing the inputs of legal processes-especially in relation to judicial ideology-however, until very recently, there have been no comparably sophisticated measures of the outputs of legal processes case outcomes. Without good measures of case outcomes-the relative placement of the substantive determination by the Court in each case-it is impossible to empirically answer questions such as whether the Supreme Court's jurisprudence in a particular area has become more conservative over time. In a recent theoretical paper, Tonja Jacobi demonstrates how the same sophisticated measures of judicial ideology that scholars use as inputs in empirical legal studies can be utilized to calculate sophisticated measures of case outcomes. 1 Jacobi actually develops three different measures, each corresponding to a different theory of judicial decision-making. This Article extends Jacobi's earlier theoretical work by empirically testing the competing measures of case outcomes. We refer to these competing measures collectively as the "Jacobi measures," for lack of a better term. This empirical analysis enables us to directly assess the plausibility of the Jacobi measures and their underlying behavioral assumptions-a vital question for empiricists and non-empiricists alike. Comparing cases, delving into their details, making fine-grained comparisons , and sifting for nuances of consistency and inconsistency is the heart of legal doctrinal analysis. Scholars establish the breadth and scope of rules and standards alike by distinguishing or reconciling competing understandings of individual cases. Imagine, then, if we had a valid and reliable mechanism of scoring case outcomes and comparing cases via a consistent, objective standard. Such scores would not replace doctrinal analysis, but they certainly would aid 1. Tonja Jacobi, Competing Models of Judicial Coalition Formation and Case Outcome Determination , 2 J. LEcAL ANALysts 411, 413 (2009) ("Currently, empirically-minded scholars have a simple measure of case outcomes and a sophisticated measure of judicial ideology available for their research. It is possible to combine these two elements to create a potentially more sophisticated measure of case outcomes."). and complement the more detailed case-by-case analysis in which law scholars have traditionally engaged. 2 In contrast to doctrinal analysis, empirical legal analysis typically reduces jurisprudential complexity, sometimes to simple binary outcomes, in order to aggregate and compare hundreds or thousands of cases in a methodologically rigorous way. Recent legal scholarship has been marked by increasing …